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14.  ABSTRACT 

The  SmartCapture,  H.264  technology  for  off-processor  video  encoding  with  Rate-Adaptive  Video  Coding  (RAVC)  extensions  is 
state-of-the-art  H.264  at  the  optimum  time  in  the  H.264  development  cycle  for  DoD  applications  such  as  UAS  video.  A  USB  stick 
embodiment  of  the  technology  is  described.  This  SmartCapture  device  incorporates  what  probably  is  the  best  available  commercial 
H.264  codec  ASIC,  a  small  processor,  and  an  NTSC  digitizer.  This  device  leveraged  commercial  development  funds  and  therefore 
the  technology  has  not  yet  been  optimized  for  a  particular  UAS  solution.  RAVC  scales  data  rate:  1)  in  the  spatial  domain,  2)  in  the 
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typical  rates  vary  from  2,5  Mbps  with  appreciable  transform  artifacts  at  lower  rates.  GOP  varies  noise  immunity. _ 
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Introduction 


The  SmartCapturc  rate  adaptive,  H.264  video  encoder  solution  for  UAS  and  other 
systems  is  believed  to  be  the  world’s  smallest  H.264  solution.  In  its  commercial  form, 
this  device  has  an  USB  memory-stick  package-style  and  USB  communications  to  with 
the  host  computer.  The  full  rate  adaptivity  of  the  H.264  standard  has  been  made  available 
to  the  user  for  on-the-fly  changes.  This  adaptivity  is  normally  only  made  available  to  the 
developer  of  the  numerous  products  for  which  the  ASIC  chip  used  was  designed. 

The  SmartCapturc  device  incorporates  what  probably  the  best  available  commercial 
H.264  codec  ASIC,  a  small  processor,  and  an  NTSC  digitizer.  This  device  leveraged 
commercial  development  funds  and  therefore  was  not  optimized  for  the  UAS  solution.  A 
better  solution  would  use  a  larger  processor,  which  could  perform  other  UAS  functions 
such  as  communications,  autopilot,  and  scan  of  a  CCD  video  chip.  A  block  diagram  is 
shown  below  in  figure  I . 
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Figure  1 .  Block  diagram  of  SmartCapture  device 

This  device  allows  for  flexible  joint  source-channel  coding,  with  data  rate  adapting  to  the 
channel.  It  also  allows  a  pre-look  at  what  will  be  possible  with  the  just  released  Scalable 
Video  Coding  (SVC)  standard  (ITU/H.264  \  ISO/lEC  MPEG-4  Part  10,  Amendment  2). 
However,  a  key  difference  is  that  SmartCapture  currently  scales  the  data  rate  of  the  video 
only  at  the  source  whereas,  the  scalable  standard  produces  a  bitstream  where  the  data  rate 
(and  quality  of  video)can  be  changed  at  any  place  in  the  communications  chain  by 
throwing  away  preselected  packets. 

Networking  software  with  RTP/RTCP,  channel  estimation,  and  adaptive  rate  control  has 
also  been  developed  and  is  described  here. 

Rate  adaptivity  can  be  done  with  NTSC  video  by  scaling:  1)  in  the  spatial  domain,  2)  in 
the  temporal  domain.  3)  in  the  encoder  fidelity  domain,  and  4)  in  the  group  of  pictures 
(GOP)  domain.  Useful  video  can  be  produced  from  32  kbps  to  4  Mbps.  In  the  spatial 
domain,  image  resolutions  of  720x480  pixels,  640x480,  352x288,  320  X  240  and  160  x 
120  can  produced.  In  the  temporal  domain,  frame  rates  of  30,  15,  10,  and  5  can  be  used. 
In  the  fidelity  domain  at  standard  resolution  typical  rates  vary  from  2.5  Mbps  to  500 
Mbps  with  appreciable  transform  artifacts  at  lower  rates.  A  typical  GOP  at  2.5  Mbps  may 
be  30  frames;  lower  GOP  values  produce  more  noise  immunity  and  higher  data  rates. 

Five  points  of  view  can  be  taken  on  the  results  of  this  development 

1 .  It  is  a  commercial  product  and  marketing  strategy.  From  the  standpoint  of 
amortizing  the  research  the  product  is  a  loss  leader  for  subsequent  product 
development. 

2.  It  is  a  device  for  evaluation  in  battlefield  video  sensors  including  UASs. 

3.  It  is  a  rapid  prototyping  facility  for  what  is  possible  with  H.264  video. 

4.  It  is  a  hardware  video  source  solution  for  Joint  Source-Channel  Coding. 

5.  It  is  a  prelook  at  the  emerging  Scalable  Video  Coding  (SVC)  standard 
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The  initial  research  was  jjerformed  under  an  AFRL  Dual  Use  Program  on  Joint  Source- 
Channel  coding.  The  development  of  the  device  itsself  was  co-funded  by  Navy  Navair  for 
potential  use  in  FireScout  UAV,  AFRiyRIGC,  and  commercial  funds.  SmartCapture  is 
under  evaluation  by  US  Army  Redstone  arsenal,  including  leadership  of  Apace 
Helicopter.  SmartCapture  is  also  being  tested  and/or  used  by  a  number  of  defense 
companies;  including  SAIC,  Booz  Allen,  Harris,  Echo,  as  well  as  the  NCIIF  (Network 
Centric  Interoperability  and  integration  Facility)  at  AFDRL/RIGC.  SmartCapture  has 
also  been  presented  to  the  Motion  Imagery  Standards  Board.  During  development  a 
presence  was  maintained  both  by  the  contractor  and  AFRL/RIGC  on  the  JVT  (Joint 
Video  Team)  between  ISO/MPEG  and  ITU/VCEG  as  a  member  of  the  US  National  Body 
of  MPEG  on  the  scalable  video  addition  to  H.264  video  standard.  (As  a  side  note.  Dr. 
Topiwala,  President  of  FastVDO,  has  been  the  Treasurer  of  the  US  National  Body  since 
March.  2004,  and  is  currently  running  for  Chairman  of  the  US  National  Body.) 

The  remainder  of  this  report  shows  the  technical  specs  of  the  SmartCapture  device,  the 
manual  for  the  device,  the  manual  for  the  commercial  software  and  a  user’s  manual  for 
the  networking  software  developed  for  AFRL.  The  interested  reader  is  referred  to  W. 

Dai;  Sachin  Patil;  Pankaj  Topiwala;  David  Hench,  Rale  Adaptive  Live  Video 
communications  over  IEEE  802. 1 1  Wireless  Networks,  SPIE  Proceedings  6696,  Sept 
2007. 
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Appendix  A.  SmartCapture  Specifications 
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Appendix  B.  Paper  Published  in  SPIE  2007 


1.  OBJECT  AND  SCOPE 

Video  is  increasingly  available  on  the  battlefield  due  to  the  advent  of  small  wireless  cameras.  These 
cameras  can  be  mounted  in  man-portable  UASs  or  in  other  battlefield  sensors.  Battlefield  communications 
are  becoming  iiKieasingly  net-centric  and  video  and  raw,  unedited  and  unprocessed  video  will  not  fit  the 
net-centric  model  since  the  uncompressed  bandwidths  are  too  high.  Currently,  bandwidth  constraints 
prevent  this  video  from  being  cfTectivcIy  shared  with  battle  commanders  or  with  intelligence  operatives.  In 
the  battle  field  scenario,  the  viewer  is  trained  and  he  could  help  in  the  trading  off  the  video  quality  he 
required  in  order  to  select  and  view  more  video.  These  factors  make  the  rate  adaptive  and  scalable  video 
essential  [  I  ]. 

The  scope  of  this  effort  is  to  develop  real-time  rate  adaptation  in  video  coding  with  link-quality  analysis. 
Include  the  ability  to  scale  down  in  rate,  resolution,  and  frame  rate. 

2.  INTRODUCTION 


This  report  presents  an  exan^lc  rate  adaptive,  live,  video  communication  system  over  IEEE  802.11 
wireless  networks,  which  integrates  channel  estimation,  rate  adaptive  video  encoding,  wireless 
transmission,  reception,  and  playback.  This  system  has  the  following  features:  I)  IEEE  802.11  wireless 
WLAN  is  used  as  the  communication  network:  2)  live  video  capturing  and  rate  control  coding  via  an 
external  USB  device;  3)  low  power  consuming  electronics;  4)  realtime  bandwidth  estimation;  5)  \idco 
streaming  conforming  to  RTF  and  RTCP  protocols. 

In  most  video  streaming  applications,  the  video  sources  are  pre-existing  (e.g.,  film  content).  In  such  cases,  a 
streaming  application  uses  the  bandwidth  estimation  to  adapt  the  sending  rate  only.  In  our  application,  the 
bartdwidth  estimation  changes  the  video  coding  rate  as  well  and  hence  the  video  quality  adapts  to  available 
bandwidth.  A  rate  adaptation  mechanism  will  be  used  between  bandwidth  estimation  and  rate  control  to 
track  the  dynamic  bandwidth  changes  and  guarantee  smooth  and  stable  video  ciKoding. 

Rate  adaptivity  is  an  important  cotKcpt  in  data  intensive  applications  such  as  video  communications.  The 
available  bandwidth  to  a  live  video  application  will  directly  impact  the  application's  performance,  i.c. 
resolution,  quality,  frame  rate,  bitratc  and  delay.  Wireless  networks  irr^sc  extra  challenges  to  bandwidth 
estimation  because  they  arc  very  sensitive  to  environmental  conditions  irKluding  low  reception  signal 
strength,  path  loss,  fading,  interference  or  connectivity  itself.  Those  effects  become  more  pronounced  when 
the  platform  itself  is  in  motion.  Live  video  communication  over  quick  transition  wireless  channel  needs  to 
update  the  available  bandwidth  every  few  seconds  to  avoid  client-side  buffer  underflows  and  to  limit  u.ser 
wait  periods  to  use  the  application.  This  implies  a  fast  convergence  tiriK  requirement.  The  bandwidth 
estimation  is  typically  taken  within  a  single  application  stream.  This  adds  one  more  requirement  that  a 
bandwidth  estimation  tool  must  be  minimally  intrusive  so  as  to  not  adversely  impact  the  application's 
performance  dunng  measuremenLs. 

Although  bandwidth  estimation  techniques  have  been  widely  studied  recently  [2.3,4,5,6  and  reference 
therein],  many  bandwidth  estimation  techniques  arc  developed  for  wired  networks  and  most  bandwidth 
estimation  tools  arc  aimed  to  network  management  and  monitoring  instead  of  real  time  video  applications. 
Accuracy  is  usually  the  main  concern  when  comparing  different  techniques  or  tools.  Live,  wireless  video 
communication  poses  different  requirements  to  the  bandwidth  estimation.  Realtime  update  becomes  the  key 
challenge.  Accuracy  therefore,  although  desirable,  is  no  longer  the  primary  concern.  In  this  report,  we 
introduce  a  method,  which  addresses  wireless  network  properties  yet  remains  practical  for  live  video 
.streaming. 

The  remainder  of  this  report  is  organized  as  follows.  Section  2  summarizes  measurement  metrics  and 
existing  channel  estimation  techniques.  Section  3  describes  the  system  infrastructure  of  the  rate  adaptive, 
live,  video  communications  system  over  IEEE  802. 1 1  wireless  network  and  the  functions  of  each  module. 
Section  4  describes  the  rate  adaptive  live  video  communication  algorithm.  Experimental  results  are 
presented  in  section  S.  Section  6  gives  cotKiusions  and  suggestions  for  future  work. 


5 


3.  RFA'IEW  OF  BANDWIDTH  ESTIMATION  TECHNIQUES 

This  section  reviews  existing  measurement  techniques  for  capacity  and  available  bandwidth. 

The  capacity  C  and  available  bandwidth  A  are  two  commonly  used  metrics,  as  they  relate  to  the  amount  of 
dau  that  a  link  or  network  path  can  deliver  per  unit  of  time.  The  capacity  is  the  maximum  IP-layer 
throughput  that  the  path  can  provide  to  a  flow,  when  there  is  no  competing  traffic  load  (cross  traffic).  The 
available  bandwidth,  on  the  other  hand,  is  the  maximum  IP-layer  throughput  that  the  path  can  provide  to  a 
(low,  given  the  path’s  current  cross  traffic  load  to  digital  communications  [4], 

Let  H  be  the  number  of  hops  in  a  path.  C,  be  the  transmission  rate  or  capacity  of  link  i,  and  Co  be  the 
transmission  rate  of  the  source,  then  the  end  to  end  capacity  C  is  determined  by  the  link  with  minimum 
transmission  rate  [4]; 

C=  min,.<,,  ,hQ  ,  (1) 

If  u,  is  the  utilization  of  link  i  (with  0  £u,  ^  I  and  Uo  =  0).  the  available  bandwidth  A  of  the  path  is 
detemuned  by  the  link  with  the  minimum  unused  capacity  [4]; 

A=minH).  j([C,(l-U|)).  (2) 

Current  active  bandwidth  estimation  techniques  can  be  divided  into  4  categories:  I )  single  packet  probing, 
like  variable  packet  size;  2)  packet  pairitrain  dispersion  probing:  3)  self-loading  periodic  streams;  4)  probe 
gap  itkmIcI  techniques. 

Variable  packet  size  measures  the  round  trip  time  RTTs  to  all  hops  on  the  path.  Jacobson’s  pathchar  [9], 
Downey's  clink  [10].  Mah’s  pchar  [I  I]  use  this  technology  to  measure  the  capacity  for  every  link  in  a  path 
[4],  These  tools  usually  require  ICMP  replies  from  the  router.  The  measurement  is  often  quite  inaccurate  as 
different  routers  may  have  different  ICMP  response  times.  Certain  network  elements  may  not  even  have  an 
ICMP  respon.se. 

Self-loading  periodic  streams  send  a  number  of  equal-sized  packets  to  the  receiver  at  certain  rate  R.  Train 
of  Packet  Pairs  (TOPP)  [12],  pathload  [13]  and  path  Chirp  [14]  probe  the  end-to-end  network  path  using 
increasing  probing  rate.  They  provide  accurate  bandwidth  estimation  for  wired  network  at  the  cost  of  long 
convergeiKC  time  and  high  intrusiveness  [6]. 

Probe  Gap  Model  techniques,  such  as  Initial  Gap  IntTcasc/Packct  Transmission  Rate  (IGI)  [15],  measure 
available  bandwidth  by  estimating  the  cros.sing  traffic  at  the  tight  link  and  by  monitoring  the  gap  changes 
after  the  packets  pass  through  the  tight  link  router.  However.  IGI  assumes  a  known  constant  capacity, 
which  is  not  valid  for  wireless  networks. 

Packet  dispersion  techniques  [4  and  references  therein),  including  packet  pair  and  packet  trains,  measure 
the  end-to-end  capacity  of  a  network  path.  Packet  pair  dispersion  sends  two  equal-sized  packets  back-to- 
back  into  the  network.  After  traversing  the  narrow  link,  the  time  dispersion  between  the  two  packets  is 
linearly  related  to  the  link  with  the  least  capacity.  Packet  train  dispersion  extends  packet  pair  dispersion  by 
using  multiple  back-to-back  probing  packets.  However,  the  concepts  for  packet  train  arc  similar  to  that  of  a 
single  packet  pair. 


Fig.  1 .  Packet  Dispersion.  The  width  of  each  link  corresponds  to  the  capacity  [4] 

Among  these  methods,  packet  dispersion  is  one  of  the  most  simple  and  mature  bandwidth  estimation 
techniques  for  wireless  networks,  thanks  to  its  fast  measurement  time  and  modest  network  load. 
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Fig.  1  illustrates  the  packet  dispersion  concept.  When  packets  of  size  L  with  initial  dispersion  Aa,  go 
through  a  link  of  capacity  Ci,  the  di.spcrsion  after  the  link  Aoui  bcconres  (4J: 


A 


nut 


max(A^,— ) 
C 


(3) 


After  packets  traverse  each  link  on  an  H  hop  end-to-end  path,  the  final  dispersion  at  the  receiver  is  [4): 


IwO.-  Jl 


C,  C,  C 


Where,  C  is  the  end-to-end  capacity.  Therefore,  the  end-to-end  path  capacity  can  be  estimated  by  C’^^L/A* 


In  wired  network,  the  capacity  of  link  C|  is  a,ssumcd  to  be  a  fixed  value.  In  wireless  network,  special 
features,  such  as  dynamic  rate  adaptation,  random  delay  between  successive  packets.  MAC  layer 
connection  backoff,  MAC  layer  ARQ.  basic  two-way  handshake  or  four-way  hand.shakc  influence  the 
capacity  insUntly.  M  Li.  M  Claypool  and  R.  Kinichi  proposed  two  packet  dispersion  measurements: 
effective  capacity  C,  and  achievable  throughput  A,  to  emphasis  these  significant  differences  between  the 
wired  and  wireless  network.  Since  transmitting  rate  in  wireless  WLAN  is  time  varying,  effective  capacity 
Cc  is  defined  as  a  function  of  the  packet  size  and  time  used  to  indicate  the  effective  transmit  rate  of  the 
wireless  network  to  deliver  network  layer  traffic  during  a  given  time  period  fb].  Replacing  Ar  by  a 
continuous  variable  T(l)  and  take  average  over  the  given  time  period  [to,  t|),  C=L/Ar  can  be  extended  to 
wireless  situation: 


(5) 


Where.  T(t)  is  the  packet  pair  dispersion  at  time  t. 

Given  discrete  packet  pair  samples  T(i)  (Ar  at  time  index  i).  the  average  effective  capacity  Q  is  [6]: 


n 


(6) 


Achievable  throughput  A|  is  the  maximum  throughput  that  a  node  can  achieve  in  contending  with  other 
existing  traffic  in  a  wireless  network.  This  is  approximated  as  the  length  of  the  packet  divided  by  the 
average  dispersion  [6]: 


A.  = 


nl. 


r.T<') 


(7) 


Where  n  is  the  number  of  samples  from  packet  pair  measurements  and  T(i)  is  the  dispersion  of  the  nth 
packet  pair. 

In  this  report,  we  will  use  capacity  or  available  bandwidth  if  not  specified  otherwise.  But  the  user  should 
keep  the  difference  mentioned  above  in  mind. 


4.  SYSTEM  INFRASTRUCTURE 

The  rate  adaptive  video  communications  system  contains  two  parts:  the  server  and  the  client  receiver.  The 
system  flow  graphs  of  both  parts  are  given  in  Fig.  2.  The  system  setup  is  illustrated  in  Fig.  3. 
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(a) 


(b) 


Fig.  2.  System  (low  diagrams:  (a)  rate  adaptive  video  encoding  streamer,  (b)  rate  adaptive  video  receiver 


(a)  (b) 


Fig.  3.  (a)  An  example  system  setup  of  the  rale  adaptive  video  communication  system;  (b)  A  close-up  look 
of  the  SmartCapturc  device  hooked  to  the  USB  port  of  the  transmitting  system 

In  Fig.  3,  the  laptop  on  the  left  is  the  receiver.  The  laptop  on  the  right  is  the  transmitter.  SmartCapturc 
device  is  hooked  to  the  USB  port  of  the  transmitter  laptop.  The  output  of  a  DVD  player  is  split  to 
SmartCapture  as  well  as  a  large  screen.  Video  is  transmitted  via  a  wireless  network.  QuickTime  is  used  to 
play  the  received  video  stream.  The  video  also  display  to  the  LCD  simultaneously.  Note  the  near 
simultaneity  of  the  live  playback  on  the  big  screen,  and  the  captured,  encoded,  transmitted,  received  and 
decoded  digital  signals. 

4.1  Video  Encoding 

Video  encoding  is  done  by  the  SmartCapturc  device  shown  in  Fig.  4.  The  SmartCapturc  device  is  an 
external  USB  device  which  converts  the  analog  video  and  audio  capture  from  VCR.  camcorder  or  video 
camera  to  H.264/AAC/MP4  stream  and  connects  the  stream  to  the  PC  or  Laptop.  It  is  today  the  world’s 
smallest  H.264  SD  encoder  board.  Other  similar  products  arc  available  (e.g..  Hauppage  USB-Live, 
http://www.hauppauge.convpages'products/’data_usblivc.html)  for  MPEG-2  and  MPEG-4,  but  no  other 
product  has  been  announced  for  H.264/AVC. 
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Fig.  4.  FastVDO  SmartC'apturc  device 


On-board  chips  digitize  and  down-sample  (if  needed)  the  analog  video  signal,  then  compress  the  video  into 
the  H.264  format.  H.264  baseline  profile  is  used  for  both  video  encode  and  decode.  The  device  can  be 
optimized  for  either  CBR  or  VBR  rate  control.  The  frame  rates  currently  support  arc  30/15/10/5  frames  per 
second.  Commands  can  be  send  from  the  PC:  to  the  device  via  the  USB  port  to  change  bitratc  on  the  fly. 
Despite  the  bitratc.  other  video  coding  parameters  such  as  resolution  (width  and  height),  frame  rate  and 
GOP  (group  of  picture)  can  also  be  changed  if  ncccs-sary.  For  more  information  about  the  SmartCapturc 
device,  please  visit  http:'7www. fastvdo.com. 

4.2  Bandwidth  Kstimation 

The  channel  estimation  module  is  a  crucial  part  of  the  whole  system.  If  the  available  bandwidth  is 
overestimated,  more  data  will  be  sent  to  the  network  than  can  be  handled,  and  cause  congestion.  If  the 
available  bandwidth  is  underestimated,  the  video  quality  suffers  from  being  compressed  at  a  lower  bitratc 
(or  framcratc)  than  necessary.  Of  course,  overcstimation  is  much  more  harmful  than  underestimation.  The 
packet  loss  rate  increases  dramatically  when  congestion  occurs,  which  puts  more  stringent  requirements  for 
error  coiKcalmcnt  and  error  robustness  at  the  decoder  side. 

Small  disruption  of  the  streaming  video  is  another  requirement  for  the  bandwidth  estimation  algorithm. 
Probes  (UDP  packets)  are  used  to  sniff  the  network  traffic.  Generally  speaking,  the  more  probes  are  sent, 
the  more  accurate  the  estimated  bandwidth.  However  too  many  probes  can  pos.sibly  delay  or  even  block  the 
video  streams.  Ideally,  the  probes  should  take  very  little  bandwidth  or  cause  no  intrusion,  meaning  there  is 
no  interference  to  the  video  stream. 

Speed,  which  is  mainly  determined  by  the  algorithm  convergence  time,  is  also  an  important  consideration. 
Environmental  conditions  generally  cause  wireless  capacity  variability  to  occur  randomly  over  short  times. 
Bandwidth  estimation  should  keep  up  with  the  change.  Wireless  communications  between  UAVs  and 
ground  .sution.  for  example,  is  more  susceptible  to  the  outdoor  environment  and  rapidly  changing 
distances.  Accurately  algorithm  with  long  converge  time  is  not  acceptable  here  since  it  docs  not  satisfy  real 
time  available  bandwidth  update.  A  video  friendly  available  bandwidth  estimation  algorithm  will  be 
described  in  section  5. 

Accuracy  therefore,  although  desirable,  is  no  longer  the  primary  concern.  The  streaming  nK'dia  applications 
change  the  sending  rate  in  quantum  steps  instead  of  doing  so  smoothly.  For  example,  in  Table  I,  any 
bandwidth  estimated  between  128Mbps  to  256Mbps  will  trigger  the  device  to  code  at  128Mbps.  While 
finer  rate  adaptation  can  be  achieved  in  a  slowly  changing  environment,  it  may  not  be  useful  in  a  rapidly 
changing  environment. 

4J  Kate  .Adaptivity 

The  estimated  available  bandwidth  is  calculated  from  the  statistics  at  the  current  instance  or  for  the  pa.st 
duration.  It  forecasts  but  doesn't  guarantee  that  the  real  available  bandwidth  during  the  following  few 
seconds  is  exactly  the  saitK.  It  can  drop  quickly  and  dramatically,  for  cxan^le  upon  observation  of  massive 
cross  traffic  or  bccau.se  of  uncertain  channel  fading.  For  these  reasons,  the  estimated  available  bandwidth 
cannot  be  u.scd  directly  to  change  the  video  coding  bit  rate  at  the  newly  estimated  rate.  Instead,  we  increase 
the  video  bit  rate  step  by  step  slowly  in  ease  the  estimated  available  bandwidth  increases.  On  the  other 
hand,  if  the  estimated  available  bandwidth  decreases,  the  video  bit  rate  should  be  dropped  immediately  to 
reflect  the  change.  Table  I  gives  an  example  how  to  change  video  bitrate  R  observing  the  available 
bandwidth  A  Table  2  shows  current  default  video  coding  parameters  given  the  video  bitratc  R. 
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4.4  Video  atrcamin|> 

The  standard  Real-time  Transport  Protocol  (RTP)  builds  on  UDP/IP.  and  provides  timing  recovery  and  loss 
detection,  to  enable  the  development  of  robust  systems.  There  are  two  parts  of  RTP:  the  data  transfer 
protocol  and  an  associated  control  protocol  (RTCP),  which  is  used  for  periodic  reporting  of  reception 
quality,  source  description  information,  and  the  information  needed  to  synchronize  media  streams  [7,8]. 
We  have  used  RTCP  packets  at  I  packet  per  second  to  communicate  from  the  streamer  to  the  receiver, 
sender  report  (SR),  and  back,  receiver  report  (RR)  to  communicate  packet  loss  and  throughput. 


Table  I.  Video  bitratc  (R)  adjusted  according  to  previous  video  bitrate  (B)  and  available  bandwidth  (A) 


B 

Kbps 

0<A<64 

64<A<128 

I28<-A< 

256 

256<=A< 

512 

5I2<=A< 

1024 

1024<=A< 

15.36 

A>=1536 

f>4 

64 

64 

128 

128 

128 

128 

128 

128 

64 

64 

128 

256 

256 

256 

256 

256 

64 

64 

128 

256 

512 

512 

512 

512 

64 

64 

128 

256 

512 

1024 

1024 

1024 

64 

64 

128 

256 

512 

1024 

1536 

1536 

64 

64 

128 

256 

512 

1024 

1536 

Table  2.  Video  coding  parameters  changed  with  bitrate 


B 

64Kbps  <=B< 

200Kbps 

200Kbps<=B< 

400Kbps 

400Kbps<=B< 

1.5Mbps 

B>1. 5Mbps 

Resolution 

160x120 

320x240 

320x240 

640x480 

Frame  rate 

15 

15 

30 

30 

GOP 

15 

15 

30 

30 

5  LIVE  RATE  ADAPTION  ALGORITHM 

Since  packet  dispersion  is  the  most  simple  and  cfTicicnt  method  to  give  quick  bandwidth  estimation,  we 
will  use  this  technique  in  our  application.  A  variable  length  packet  train  of  UDP  packets  without  RTP 
encapsulation  will  be  sent  to  the  receiver  to  estimate  the  available  bandwidth.  The  size  of  each  probe  packet 
is  1500  bytes,  or  12000  bits.  The  length  of  the  packet  train,  however,  is  determined  by  the  previous 
estimated  available  bandwidth  such  that  5%  of  the  available  bandwidth  is  used  for  probing.  When  the 
available  bandwidth  is  extremely  low,  iki  probe  will  be  sent  until  the  application  decides  it  is  time  to 
incrca.se  the  video  coding  and  sending  rate,  which  for  example,  can  be  triggered  by  no  video  packet  loss  for 
several  seconds.  Let  subscript  i  indicate  the  lime  index  with  a  one  second  interval,  given  the  available 
bandwidth  at  lime  i.  A,,  the  length  of  packets  N  at  time  i  is; 

N,=ro.05  ♦  A,.,  /(1 500x8)1= Fa,.,  /24000o1  for  i  =1,2,3...  (8) 

The  achievable  throughput  is  calculated  by  the  receiver  as  A,  =  ,T))  =  l2000n/(2^-i  ..T,). 

where  L  is  the  probe  packet  size  in  bit  and  n  <=  N,.  When  pan  or  all  packets  are  lost,  n  is  less  than  N,.  The 

Ti  is  the  time  dispersion  between  packet  i-l  and  i,  for  i=l,2 . n,  which  can  be  measured  by  recording  the 

arriving  time  of  packet  i- 1  and  packet  i. 

Although  fast  and  minimal  intrusion  available  bandwidth  can  be  estimated  by  packet  dispersion  method 
used  in  our  application,  it  can  over  estimate  the  available  bartdwidth.  For  example,  when  the  length  of 
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packet  train  N  is  small,  it  cannot  always  capture  the  cross  traffic  in  the  network  thus  the  estimation  is 
higher  than  the  actual  value.  This  overshooting  can  be  avoided  by  take  the  receiving  video  data  rate  into 
consideration.  If  the  receiving  video  data  rate  is  less  than  the  estimated  bandwidth,  the  smaller  value  will  be 
used.  The  receiving  video  data  rate  can  be  calculated  from  the  RTCP  sender  report  (SR)  and  receiver  report 
(RR).  Let’s  define  R«a.,to  be  the  video  sending  bit  rate  between  two  consecutive  RTCP  SR  at  time  i  and  i- 
R  TSV4  to  video  receiving  bit  rate  received  by  the  receiver,  and  L,  to  be  the  loss  fraction  at  time  i, 
which  will  be  available  from  RTCP  RR  at  time  i.  Then  we  have  R  t„4  ■  R«d,i  *  L,,  R^i  can  be  easily 
calculated  by  the  sender. 

Other  terms  used  in  this  session  are  defiiKd  here.  B,  is  the  device  coding  bitrate  at  time  i.  The  live  rale 
adaptive  algorithm  is  summari/ed  as  follows; 

Step  I:  Initialization: 

1.1  Set  Ao“  1  Mbps,  setNp=rAo^4(H)00l»5.  sctR,t»,,«R«,d.i“lH*0.  Bo“5l2Kbps 
Step  2:  Packet  loss  detection; 

2.1  SetL,=0 

2.2  Sender  sends  RTCP  SR,  and  updates  Rndj 

2.3  Receiver  receives  SR  i  and  responses  with  RR, 

2.4  If  sender  gets  RTCP  RR  j. 

2.4.1  Update  L,  from  RR, 

2.4.2  Calculate  R  w,j=  R.~i.  *  L| 

2.4.3  Set  SendProbcPackets  *  true 

2.5  Else  if  sender  cannot  get  RTCP  RR  ( 

2.5.1  Set  R  =0 

2.5.2  Set  A,=  Ah /2 

2.5.3  Set  SendProbcPackets  *  false 
Step  3:  Achievable  throughout  estimaUQp; 

3.1  If  SendProbcPackets 

3.1.1  Calculate  N,  -  f  Ah  /2400001  ,  set  A, »  0 

3. 1 .2  Send  N,  probes  to  receiver 

3. 1 .3  Calculate  A,  using  equation  (7) 

3. 1 .4  Receiver  sends  back  A,  to  sender 

3.1.5  If  sender  gels  feedback,  update  A, 

Step  4;  A,^ validation: 

4.1  lfL,>0,  A,  *min(A,.  R  ) 

Step  5:  Video  rate  adjustment; 

5.1  Update  B,  according  to  Table  I 

5.2  Send  change  parameter  command  to  SmartCapture  device  according  to  Table  2 
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6  EXPERIMENTAL  RESULTS 


NodeB 


Fig.  5.  Lab  setup 

Three  computers,  labeled  as  A.  B  and  C  shown  in  Fig.  5  arc  wirelessly  connected  via  an  802.1  Ig  network. 
Node  A  serves  as  the  video  streamer.  Node  C  is  the  video  receiver.  Node  B  is  used  to  add  load  to  the 
network.  The  operation  system  Node  A  uses  is  open  SUSE  10.2  with  2.6  Linux  kernel.  The  wireless 
network  card  used  in  node  A  is  Intel  PRO/wircless  2200BG.  The  operation  system  on  Node  B  and  Node  C 
is  Windows  XP,  the  wireless  adaptors  used  in  node  B  and  C  is  D-Link  DWL-GI22.  The  router  we  use  is 
Dlink  DI-524,  which  supports  both  802.1 1  b  and  g.  We  u.sc  iperf  ( 17]  to  add  load  and  as  an  external,  third 
party  bandwidth  estimation  tools.  The  network  trafTic  is  categorized  as:  video  stream  (from  A  to  C),  RTCP 
and  probe  (two-way  between  A  and  C)  and  cross  traffic  (sending  by  iperf  from  B  and  C). 

6.1  Estimate  the  available  bandwidth  with  no  load 


Fig.  6.  Available  Bandwidth  as  estimated  by  a  popular  tool  called  iperf.  Test  duration  is  100  seconds 

The  popular  tool  iperf  [16]  is  used  to  estimate  the  bandwidth  and  add  load  to  the  network.  Iperf  u.scs  bulk 
data  transfer  method  to  saturate  the  channel.  In  Fig.  6,  ipef  is  used  only  to  measure  the  effective  capacity. 
The  capacity  is  not  a  constant  in  this  test  because  it  is  very  sensitive  to  environmental  conditions  including 
low  reception  signal  strength,  path  loss,  fading,  interference  of  other  wireless  devices  or  connection  itself 
Further  more,  those  conditions  could  possibly  trigger  the  rate  adapt  algorithm  embedded  in  the  router  and 
make  it  switch  to  a  lower  transmitting  rate.  That  can  explain  the  sudden  drops  on  the  estimated  available 
bandwidth.  Sometimes,  it  drops  dramatically.  The  test  lasts  for  100  seconds.  The  average  capacity  over  the 
100  seconds  is  10.3  Mbps. 
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Fig.  7.  Available  bandwidth  as  estimated  by  FaslVDO.  Test  duration  is  100  seconds 

Fig.  7  shows  the  result  estimated  by  our  application.  No  other  traffic  present  except  the  probes.  The  test 
lasts  for  100  seconds  with  an  estimated  capacity  around  9.4Mbps.  This  is  about  0.9  Mbps  less  than  as 
estimated  by  iperf.  Note  that  unlike  iperf.  which  uses  full  bandwidth  in  its  bulk  data  transfer,  FastVDO’s 
method  uses  only  about  5%  of  the  available  bandwidth.  This  approach  is  much  more  traffic  friendly.  A 
running  streaming  application  cannot  afford  to  run  iperf,  or  indeed  any  intrusive  bandwidth  estimation 
approach  which  u.ses  a  significant  fraction  of  the  bandwidth  for  measurement.  Also,  FastVDO’s  approach  is 
real-time,  in  which  estimation  time  is  less  than  30  ms. 

6.2  Estimate  the  available  bandwidth  with  and  without  load 


Fig.  8.  Aggregate  bandwidth  with  injected  load  and  estimated  available  throughput 

In  Fig.  8  the  test  duration  is  150  seconds.  The  square  purple  line  is  the  load  added  to  the  network  by  iperf. 
During  31-60  seconds,  the  load  is  8  Mbps.  During  101-130  seconds,  the  load  is  10  Mbps.  The  diamond 
blue  line  indicates  the  estimated  achievable  throughput  by  FastVDO.  The  triangle  yellow  line  is  the 
aggregated  bandwidth 

63  Rate  adaptive  video  streaming  with  and  without  load 

In  this  section,  load  will  be  added  to  the  network  to  investigate  the  adaptivity  of  our  application.  Available 
bandwidth  is  updated  every  one  second,  which  is  believed  to  be  adequate  to  the  system  real  time  update 
requirement. 
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Fig.  9.  Estimated  available  bandwidth  and  the  bandwidth  used  by  probing 

Fig.  9  illustrates  that  our  method  is  nonintrusivc.  In  Fig.  9.  cross  traffic  is  added  to  make  the  available 
bandwidth  have  a  large  dynamic  rage.  This  figure  shows  the  bandwidth  used  by  probing  is  changing  with 
the  previously  estimated  available  bandwidth.  It  is  about  5%  of  the  available  bandwidth.  When  the 
bandwidth  is  extremely  low  and  packet  loss  is  detected,  no  probes  will  be  sent.  All  the  available  bandwidth 
will  be  used  by  the  video  data.  The  “Probe”  curve  shows  the  actual  probe  data  rate. 
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Fig.  10.  Estimated  available  bandwidth  and  SmartCapturc  device  bitrate  .settings 

In  Fig.  10,  various  cross  traffic  arc  added  to  the  network  from  41  second  to  120  second  to  test  the  behavior 
of  our  rate  adoption  algorithm  The  dotted  line  is  the  estimated  available  bandwidth.  The  solid  line  is  the 
device  settings.  Whenever  the  bandwidth  is  drop,  the  device  coding  rate  corresponds  immediately  by 
Jumping  down  to  a  smaller  value.  The  device  rate  recovers  one  step  up  each  time.  For  example,  from  t=  96 
to  114  second,  the  bandwidth  oscillates  frequently.  Ambitious  rate  increasing  will  make  the  network 
congestion  even  worse.  And  aggressive  rate  change  will  cause  the  rate  control  algorithm  embed  in  the 
device  work  inefficiently.  Our  application  uses  smooth  rate  increase  so  that  the  device  has  a  smooth 
transition.  Sharp  rate  deduction  is  used  so  that  the  data  sending  rate  is  always  below  the  available 
bandwidth. 

Fig.  1 1  shows  the  visual  quality  at  different  bitrate.  framerate  and  group  of  picture  (GOP).  The  input  is  the 
NTSC  analog  signal.  The  video  is  then  down  sampled  to  320x240.  The  first  frame  is  coded  as  I  frame. 
Although  we  can  tell  difference  for  the  grass,  the  visual  quality  of  the  main  objects,  such  as  vehicles, 
building  and  trees  in  each  picture  arc  reasonably  consistent  and  acceptable.  That  is  the  advantage  of  reduce 
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framcratc  at  low  bitrate.  Considering  that  UDP  is  a  unreliable  transfer  protocol,  the  GOP  value  changes 
with  framcratc  so  that  we  can  always  refresh  the  I  frame  every  one  second  to  reduce  error  propagation.  The 
resolution  can  also  be  changed  if  necessary.  However,  frequent  change  of  resolution  is  not  suggested.  If  the 
sequence  parameter  set  (sps)  and  picture  parameter  set  (pps)  packets  arc  lost,  the  decoder  could  not  be 
informed  of  the  resolution  change,  which  will  possibly  make  the  decoder  crash.  Possible  solution  is  send 
sps  and  pps  over  a  reliable  connection. 


7  CONCLUSION 

In  this  report,  effective  capacity  and  achievable  throughput  arc  used  as  metrics  to  measure  the  available 
bandwidth  of  the  wireless  network.  We  use  a  packct-dispcrsion-bascd  algonthm  to  estimate  the  available 
bandwidth.  The  dispersion  rate  may  be  larger  than  the  available  bandwidth  if  the  number  of  packets  is  not 
big  enough  to  capture  the  cross  traffic.  To  overcome  the  overeslimation.  this  algorithm  uses  the  information 
such  as  loss  fraction  and  receiving  data  rate  from  RTCP  packet  to  validate  the  estimation.  This  realtime 
algorithm  is  no  intrusion,  thus  video  friendly.  Video  encoding  is  done  by  the  external  USB  SmartCapturc 
device,  thus  the  system  requirement  for  the  streamer  is  low.  Tor  example,  the  CPU  usage  is  around  ]%  in 
the  system  with  Intel  Pentium  M  2.0G  CPU.  By  design  the  whole  system  is  easy  to  migrate  to  mobile 
devices. 


Fig.  1 1.  The  first  frames  for  different  video  coding  settings.  From  top  to  bottom,  from  left  to  right: 
5I2Kbps  %  30fps.  GOP  =  30, 256Kbps  @  15fps.  GOP  -  15,  128Kbps  @  lOfps.  GOP  =  10,  64Kbps  @ 
5fps.  GOP  =  5 
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